The invention relates generally to data communication and signal processing methods and apparatus, and in particular to a method and apparatus for reliably and efficiently encoding and decoding sequences of image data, for example, that transmitted over a telephone communications channel.
The transmission of sequences of images, and in particular sequences of naturally occurring images such as those represented by a television signal, has been the subject of a significant amount of investigation. Typically, investigators have relied upon the highly redundant nature of successive images in the sequence and have often modeled the image data as a Markov process with a correlation coefficient close to unity. The three-dimensional Markov model provides a motivation for utilizing differential pulse-code-modulation (DPCM) and transform coding techniques to take account of the interframe redundancy.
By analyzing the nature of typical moving video, it is easy to become convinced that the principal change occurring between successive frames is the inhomogeneous motion of the objects within the frame. It has also been recognized that an accurate apparatus and method of estimating and compensating for this spatially dependent motion enables the construction of an interframe data compression method and apparatus which can have substantially better performance than can be achieved by sending a signal representative merely of the difference between successive frames.
As a result, various motion compensating coding methods and apparatus have been developed. These systems typically are either receiver-based motion compensation systems or transmitter-based motion compensation systems. In the receiver-based motion compensation system, the receiver makes a prediction as to the motion and compensates the previous frame for the expected motion. The transmitter, operating in the same manner, then sends only an error signal describing what must be done at the receiver in order to correct the receiver predicted frame. The error signal is typically coded to reduce its bandwidth.
For a transmitter-based motion compensation system, the motion estimation process occurs only at the transmitter. Displacement vectors are generally determined over various regions of the image and this data is then transmitted to the receiver along with an error information data signal. At the receiver, the compensation process is performed on the previously coded image first using the motion information provided by the transmitter. The error signal data provided by the transmitter is then added to the thus compensated receiver image in order to maintain picture quality.
There is thus typically provided for a transmitter-based motion compensation system a plurality of displacement vectors, and in at least one preferred embodiment, each vector is associated with a specific region or block of the image. The blocks are typically non-overlapping and have, for example, a size of eight picture elements (pixels) by eight picture elements. Various methods have been employed for encoding the motion compensation data associated with each of the blocks. Hinman, in his co-pending application U.S. Ser. No. 740,898, filed June 3, 1985, the contents of which are incorporated herein, in their entirety, by reference, describes a lossy coding method for encoding the motion-compensation displacement information.
Many methods have also been employed for encoding the error information data signal in a transmitter-based motion compensation system. In the above-identified Hinman application, a lossy compression method and apparatus are disclosed. While these methods are highly advantageous, and provide excellent results, it is nevertheless desirable to further improve the compression of the data information and thereby enable high quality image reproduction using still less channel bandwidth. It is further desirable to provide better control over the data transmission by controlling, for example, the bit rate associated with the image.
Often, during a scene change, for example, there exists substantial information to be transmitted, so that during a single frame time, insufficient bandwidth is available to transmit all of the information. Accordingly, various methods have been implemented to selectively limit the number of bits of information transmitted over the channel. One of these methods, described in Ericsson, U.S. Ser. No. 001,326, filed Jan. 7, 1987, entitled "A Method and Apparatus for Efficiently Coding and Decoding Image Sequences," employs quad-tree coding in connection with the transmission of portions of a transform coefficient data set. The quad-tree coding advantageously provides a more graceful degradation of the image during heavy motion or scene changes. The contents of the Ericsson, U.S. Ser. No. 001,326, filed Jan. 7, 1987, are incorporated herein, in their entirety, by reference.
It is therefore an object of the present invention to transmit sequences of images over a communications channel using relatively low bandwidth, and providing high reliability and fidelity. Another object of the invention is to control the number of bits employed to transmit each image of a sequence of images while providing for graceful degradation of the image during a scene change or during periods of heavy motion. Other objects of the invention are a motion compensation encoding and decoding method and apparatus which reliably transmit and receive an accurate estimate of the displacement of the pixels of a scanned image in a sequence, and an improved motion estimation encoding and decoding method and apparatus which enable real-time, accurate determination of regional displacement in an image transmission device.